per 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 5 
G06F 15/02, H04N1/00 



Al 



(11) International Publication Number: 
(43) International Publication Date: 



WO 93/14458 

22 July 1993 (22.Q7;93) 



(21) International Application Number: PCT/US92/1 1288 

(22) International Filing Date: 23 December 1992 (23.12.92) 



(30) Priority data: 
819,390 



10 January 1992(10.01.92) US 



(71) Applicant: EASTMAN KODAK COMPANY [US/US]; 

343 State Street, Rochester, NY 14650 (US). 

(72) Inventors: SCHLACK, Carl, William ; 80 Hollyvale Drive, 

Rochester, NY 14618 (US). FLYNN, James, Terrence ; 
137 Selbourne Chase, Fairport, NY 14450 (US). SOPER, 
Jay, Badger ; 236 Creighton Lane, Rochester, NY 14612 
(US). CORL, Kenneth, Grant ; 185 Elmore Road, Ro- 
chester, NY 14618 (US). GABORSKI, Roger, Stephen ; 
30 Cambric Circle, Pittsford, NY 14534 (US). PHIL- 
BRICK, Robert, Hadley ; 25 Crestway Lane, Rochester, 
NY 14612 (US). 



(74) Agent: ARNDT, Dennis, R.; 343 State Street, Rochester; 
NY 14650-2201 (US). 



(81) Designated States: AU, JP, KR, European patent (AT, BE, 
CH, DE, DK, ES, FR, GB, GR, IE, IT, LU, MC, NL, 
PT, SE). 



Published 

With international search report. 



(54) Title: IMAGE-BASED ELECTRONIC POCKET ORGANIZER 




(57) Abstract 

An electronic organizer is provided that incorporates an internal electronic scanner and a touch sensitive display screen to 
enter text and image data. The internal electronic scanner permits both machine generated text and image data to be scanned and 
directly entered into the electronic organizer. Hand-printed text data is also entered directly via the touch sensitive display screen 
using a stylus or pen. The scanned machine generated text, the scanned image data and the hand-printed text can either be pre- 
served as an image-oriented bit map, or optical character recognition routines can be applied to the data to identify characters 
and convert the identified characters to computer coded text data. Data entered into the electronic organizer is arranged in a rela- 
tional database format, which permits the operator to quickly and easily enter and retrieve related information between a number 
of different databases with a minimal amount of effort. A small document transport mechanism is provided to aid in the scan- 
ning of small size documents. 
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IMAGE-BASED ELECTRONIC POCKET ORGANIZER 



Field of the Invention 
The invention relates in general to compact 
electronic organizers that are capable of storing 
information (for example schedule information , telephone 
numbers, memos, etc.) commonly used by individuals on a 
day-to-day basis. More specifically, the invention is 
related to a compact electronic pocket organizer that is 
capable of entering and storing personal information in 
the form of both text and image data in a manner that 
permits the ' information to be quickly and easily 
correlated between a plurality of databases when entered 
and retrieved. 

Background 

A wide variety of compact electronic organizers are 
currently available that permit an operator to enter 
various information related to the operator's personal 
schedule and business contacts. For example, the WIZARD 
(Tm) series of hand-held electronic organizers available 
from the Sharp Electronics Corporation includes a 
keyboard and display that permits the operator to enter 
schedule information, telephone numbers and memos for 
later review and retrieval* The operator can also create 
a business card file in which information related to a 
number of business cards can be stored, sorted and 
retrieved . 

A primary drawback of the type of electronic 
organizer described above is the method in which 
information is entered and subsequently retrieved by the 
operator. Specifically, the operator is forced to use a 
very small keyboard to enter text data as the electronic 
organizer is designed to be of a compact or "pocket" 
size. Many individuals find the small keys on the 
keyboards to be difficult to use and may therefore 



experience a large number of errors when trying to enter 
or retrieve information. 

Attempts have been made to overcome the problem of 
small keys by having the operator use an input stylus to 
activate the keys or buttons on the keyboard. While the 
stylus does make it easier for the operator, to select a 
desired key, the overall data entry operation using the 
stylus is very slow as the operator must use a "hunt and 
peck" approach to enter data. Thus, the transfer of data 
from various source materials into the organizer, for 
example the transfer of name, address, company name and 
telephone number from a business card to a business card 
file within the organizer, becomes a very tedious and 
time consuming operation. 

Another disadvantage in the method of entering data 
in conventional organizers is that, in many instances, 
the same data must be re-entered in a number of different 
databases. For example, it may be desirable to include 
identical information concerning a particular individual, 
such as the individual's name, in a telephone directory 
file, a business card file and a memo file* Conventional 
organizers do not provide a mechanism to relate data 
between databases. Thus, the user of the organizer is 
forced to re-enter the same name information in each of 
the desired files, thereby increasing the number of key 
strokes required and the probability that a data entry 
error will occur. 

A further drawback of currently available electronic 
organizers "*is the inability to correlate various 
information from different databases for easy access and 
retrieval. For example, an operator using the electronic 
organizer's schedule function mode of operation may find 
that an appointment is indicated for a particular- 
individual at a certain time. The operator may wish to 
contact the noted individual by telephone to cancel the 



scheduled appointment. To obtain the individual's 
telephone number, the operator must exit the schedule 
function node and enter a telephone directory function 
mode. The operator must then search ' a telephone 
directory file for the individuals name to obtain the 
telephone number. Thus, the operator is . required to 
enter a number of commands to switch from one discrete 
mode of operation to another to obtain the desired 
information. The requirement to use multiple discrete 
modes to retrieve information is timing consuming and 
tedious. 

In view of the above, it is an object of the present 
invention to provide an electronic organizer that 
incorporates a user interface that reduces the amount of 
effort required to enter and retrieve both text and image 
data into the electronic organizer. It is a further 
object of the invention to provide an electronic 
organizer with the ability to correlate and relate 
information between several different databases to permit 
the operator to quickly and easily enter and retrieve 
related information with a minimal amount of effort. 
Other objects and advantages of the invention will become 
apparent from the detailed description of the best mode 
of practicing the invention provided below. 

Summary of the Invention 
The present invention provides an electronic 
organizer incorporating an easy to learn and use 
interface that includes an internal electronic scanner 
and a touch sensitive display screen to enter text and 
image data. The internal electronic scanner permits both 
machine generated text and image data to be scanned and 
directly entered into the electronic organizer, thereby 
reducing the number of manual data entry operations 
required by the operator. Hand-printed text data is also 



entered via the touch sensitive display screen using a 
stylus or pen. The scanned machine generated text, the 
scanned image data and the hand-printed text can either 
be preserved as an image-oriented bit map f or optical 
character recognition routines can be applied to the data 
to identify characters and convert the. identified 
characters to computer coded text data. Data entered 
into the electronic organizer is arranged in a relational 
database, which permits the operator to quickly and 
easily enter and retrieve related information between a 
number of different databases with a minimal amount of 
effort. A small document transport mechanism is also 
provided to assist in the scanning of small documents. 

Brief De gerintion of the Drawing 

With the above as background, reference should now 
be made to the following detailed description and the 
accompanying drawings, in which: 

Fig. 1 is a top perspective view of an electronic 
organizer in accordance with the present invention; 

Fig. 2 is a perspective view of the bottom of the 
electronic organizer shown in Fig. 1; 

Fig. 3 is a schematic diagram of a small document 
transport mechanism employed in the electronic organizer 

shown in Fig. 1; 

Fig. 4 is a schematic representation of the layout 
of a scanning unit in the electronic organizer 
illustrated in Fig. 1; 

Fig. 5 is a schematic block diagram of the 
electrical operating system of the electronic organizer 
illustrated in Fig. 1; 

Fig. 6 illustrates the display of information on a 
display unit of the electronic organizer illustrated in 
Fig. 1 on power up; 
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Fig. 7 illustrates the display of a daily schedule 
screen on a display unit of the electronic organizer 
illustrated in Fig. 1; 

Fig. 8 illustrates the display of information 
5 related to one of the fields of the daily schedule 
illustrated in Fig. 7; 

Fig. 9 illustrates the overlay of a Find function 
option screen over the display illustrated in Fig. 8; 

Fig. 10 illustrates the overlay of a Find screen 
10 over the display illustrated in Fig. 8; 

Fig. 11 illustrates the display of information 
retrieved from a memory card located in a memory card 
expansion slot of the organizer illustrated in Fig. 1; 

Fig. 12 illustrates the display of a calculator tool 
15 screen in a Tool function mode of operation; 

Fig. 13 illustrates how data can be correlated using 
a relational database in the organizer illustrated in 
Fig. 1; 

Fig. 14 illustrates the display of a virtual 
20 alphanumeric keyboard on the display unit of the 
organizer illustrated in Fig. 1; 

Fig. 15 illustrates the entry of hand-printed text 
information using the pen input unit of the organizer 
illustrated in Fig. 1; 
25 Fig. 16 illustrates a database record file that is 

displayed on the display unit of the organizer 
illustrated in Fig. 1; 

Fig. 17 illustrates the blocking of identified text 
fields by the system shown in Fig. 5; 
30 Fig. 18 is a flow diagram illustrating the operation 

of the scanner unit and digital signal processor 
illustrated to scan an image; 

Fig. 19 is a flow diagram illustrating the entry of 
data in the Text input window illustrated in Fig. 15 
35 using the pen input unit shown in Fig. 1; 
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Fig. 20 illustrates an accessory docking station 
that is used in conjunction with the organizer 
illustrated in Fig. 1; and 

Fig. 21 illustrates an organizer in accordance with 
5 the invention that includes an electronic camera unit. 

ffodes of Carrying Out the Invention 
A top perspective view of a hand-held electronic 
pocket organizer according to the present invention is 
illustrated in Fig. 1. The organizer includes a main 
10 unit 10 , a battery power unit 12 releaseably coupled to 
the main unit 10, a high resolution touch sensitive 
electronic display panel 14 located on a top surface of 
the main unit 10 , a pen input unit 16, memory card 
expansion slots 18 located in the main unit 10, scanner 
15 control start and stop buttons 20-21, and a main unit 
power ON/OFF switch 22. A pen holder slot 24 is also 
located on the main unit 10 to hold the pen input unit 16 
when it is not in use. 

The operator interacts with the main unit 10 through 
20 the use of the pen input unit 16 and the touch sensitive 
electronic display panel 14. Various overlay screens or 
"windows" are displayed on the display panel 14 and the 
operator touches the pen input unit 16 to the display 
panel 14 at specified locations to perform various 
25 functions such as data entry --including hand-printed 
text entry and virtual alphanumeric keyboard operations-- 
and organizer navigational operations —i.e. moving from 
one organizer function to another— as will be described 
in greater detail below. 
30 The display panel 14 preferably includes a resistive 

type pen input material that overlays a liquid crystal 
display having a pixel count of at least 640 X 200 in a 
text mode- of operation and 320 x200 in a graphics mode 
of operation when displayed at 72 dpi. The resistive pen 



input material consists of several layers of transparent 
materials which are fabricated such that the application 
of pressure by a pen shaped object will result in a 
voltage being generated and measured that is spatially 
proportional to the incident pens physical position. The 
use of such resistive type touch sensitive pen input 
devices in electronic digitizer tablets is well known in 
the art. It will be understood, however, that the term 
"touch sensitive" is not intended to limit the display 
panel 14 to a resistive type unit that requires physical 
contact. For example, touch screens that use light beams 
to locate the position of an object that passes through 
a plane located above the actual display surface can also 
be utilized for the display panel 14. Other types of 
displays may also be used, provided that the operator can 
selectively enter information by simply touching or 
pointing to prespecified areas of the display unit. 

As shown in Fig. 2, the organizer also includes an 
imaging window 26 for a linear electronic scanner unit 
incorporated within the main unit 10, a small document 
transport mechanism platen 28 that extends from the main 
unit 10 during the operation of a small document 
transport mechanism incorporated within the main unit 10, 
a speaker unit 30 which is used to generate acoustic 
telephone dial tones, and external power connectors 32 • 
Front transport wheels or rollers 34 are preferably 
located adjacent to the imaging window 26 of the linear 
electronic scanner unit and rear transport wheels 36 are 
provided at the opposite end of the organizer. The front 
and rear transport wheels 34, 36 permit the organizer to 
be rolled over the surface of materials to be scanned 
including, for example, materials having machine 
generated text — i.e. printed materials such as telephone 
directory listings — and images — i.e. photographs or 



graphics — for direct entry into the memory of the 
organizer. 

The scanning of small documents is a problem common 
in conventional hand-held type scanning devices , as it is 
difficult to properly locate and hold a small document in 
place as the scanner is passed over the small document. 
Thus, conventional hand-held type scanning devices may 
require that small documents, such as business cards, be 
secured to a surface with tape or some other mechanism 
prior to performing a scanning operation. The small 
document transport mechanism incorporated in the main 
unit 10 solves the problem of scanning small documents by 
providing a mechanism for grasping and transporting small 
documents past the imaging window 26 of the linear 
scanning unit. 

As shown in Fig. 3, the front transport wheels 34 of 
the organizer are preferably connected to a drive axle 
50, which in turn is coupled through a pinch roller drive 
gears 39 to a document pinch roller 41. The drive axle 
50 is also coupled via encoder gears 43 to an optical 
encoder unit 45. 

The small document transport mechanism platen 28 
shares a common pivot axis (at either end) with the drive 
axle 50. Tension springs 47 are employed to pull the 
platen 28 up against the bottom of the document pinch 
roller 41. The document pinch roller 41, the transport 
wheels 34 and the drive axle 50 are mounted on a frame 51 
that is spring loaded within the main unit 10 to permit 
the frame 51 to be extended or lowered from the bottom 
surface 53 of the main unit 10 during operation of the 
small document transport mechanism 38. In the lowered 
position, the platen 28 and the document pinch roller 41 
form a nip into which a small document is inserted for 
scanning. 
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In operation, a small document is placed in the nip 
of the small document transport mechanism 38 and the 
organizer is rolled over a smooth surface. The rotation 
of the front transport wheels 34 causes the drive axle 50 
5 to rotate, which in turn causes the document pinch roller 
41 to rotate. The pinch roller 41 subsequently drives 
the small document past the imaging window 26 of the 
linear electronic scanning unit. The illustrated small 
document transport mechanism 38 is particularly well 

10 suited for use in the organizer, as it does not require 
an electrical motor to transport the small document and 
therefore does not drain energy from the battery unit 12. 

It should be noted that, in the illustrated 
embodiment, the small document is actually scanned in the 

15 opposite direction than a large document. The organizer 
is moved from left-to-right across a page to scan a large 
document. The small document, however, transports the 
small document past the scanning window from right-to- 
left. Thus, compensation for the change in the scanning 

20 direction must be made either when data is initially 
stored or a translation operation on the data must be 
performed prior to the display of the data on the display 
panel 14. 

The optical encoder unit 4 5 generates encoding 
25 signals in accordance with the rotation of the axle 50 
that correspond to the movement of the organizer over a 
surface. The encoding signals produced by the optical 
encoder unit 45 are used to clock the operation of the 
linear electronic scanning unit so that a line of image 
30 data is generated as the organizer is moved a 
predetermined distance over the surface in a manner well 
known in the hand-held scanning art and need not be 
described detail. The encoding signals are also used by 
a central processing unit (CPU) , located on a motherboard 
35 48 within the main unit 10 of the organizer, to monitor 
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the speed of the scanning operation* The central 
processing unit preferably generates a warning signal if 
the scanning speed is exceeding a predetermined limit. 
The warning signal can be either an audio warning signal 
generated by activating the speaker unit 30 and/or a 
visual warning signal displayed either on the display 
panel 14 or on a separate LED indicator provided on the 
main unit 10. In either case, the warning signal is 
preferably of a type that provides feedback to the 
operator to warn the operator when the scanning speed 
limit is being approached, for example by increasing the 
frequency of the audio warning signal, changing the color 
of the visual warning signal, or by flashing the visual 
warning signal at varying frequencies. The visual or 
audio feedback permits the operator to reduce the 
scanning speed before the predetermined speed limit is 
exceeded. 

The layout of the linear electronic scanning unit 
within the main body 10 of the organizer is illustrated 
in Fig. 4. The linear electronic scanning unit includes 
a light source 40, a mirror 42, a lens array 44, and a 
linear electronic image sensor 46. In operation, the 
image being scanned is reflected by the mirror 42 to the 
lens array 44, which in turn focuses the image on the 
linear electronic image sensor 46. The linear electronic 
image sensor 46, with its associated control circuitry, 
converts the image to digital image data in a 
conventional manner. The digital image data is then 
supplied to a digital signal processing unit (DSP) 
located on the motherboard 48 in the form of a bit map. 

Fig. 4 also illustrates the preferred location of 
the card expansion slots 18 with respect to the 
motherboard 48. The card expansion slots 18 accept 
memory cards that can either be used to expand the system 
memory of the organizer or to hold special software 
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application programs or database packages. The card 
expansion slots 18 are preferably configured to hold 
memory cards that conform to the standards established by 
the Personal Computer Memory Card International 
Association (PCMCIA), including cards conforming to the 
Execute-In-Place (XIP) standard, although memory cards 
utilizing other configurations could be utilized. Memory 
cards that could be employed in the invention are 
currently available from the Maxell Corporation of Fair 
Lawn, New Jersey and have memory capacities on the order 
of one megabyte. 

A basic electrical schematic block diagram of the 
organizers operating system is illustrated in Fig. 5. 
The central processing unit (CPU) 60 (for example a F8680 
processor available from Chips & Technology Corporation) 
and the digital signal processing unit (DSP) 62 (for 
example a TS350C51 processor available from Texas 
Instruments Corporation) mentioned above are coupled to . 
a bus 64. System memory is provided by a one megabyte 
capacity random access memory (RAM) unit 66 and a two 
megabyte capacity read only memory (ROM) unit 68. As was 
mentioned above, additional memory can be provided by 
inserting memory cards in the card expansion slots 18 
which are also coupled to the bus 64. The CPU 60 
controls the overall operation of the organizer, while 
the DSP 62 works in conjunction with the CPU 60 to 
support processing operations related to scanned data. 

The organizer preferably supports two modes of 
scanning operations, namely, a Text Mode of operation 
(default mode) which is used to scan machine generated 
text images and a Photo Mode of operation (user selected) 
which is used to scan high resolution images such as 
photographs or graphics. Images processed in the Text 
Mode of operation are converted to one bit/pixel by 
thresholding and the image pixels are packed eight bits 
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per byte. The images are preferably compressed by a 
CCITT GIII/IV (Committee Communications Internationale de 
Telephone et Telegraph Group III and IV) lossless method. 
Text Mode images may be displayed directly on the display 
5 unit 14, or an optical character recognition algorithm 
(OCR) can be applied to the image to convert the text 
image data to computer coded text data, e.g. ASCII. 
Images in the Photo Mode of operation are processed by an 
error diffusion method wherein the images are converted 

10 to one bit/pixel by distributing the gray level error 
into the surrounding pixels. The pixels are packed eight 
bits per byte and the images are compressed using a 
lossey or lossless method, for example, JPEG (Joint 
Professional Engineering Group) algorithm. The Photo 

15 Mode of operation optimizes the quality of scanned 
photographic images as for display on the display unit 
14. 

Conventional OCR algorithms can be employed by the 
DSP 62 to identify text data in the Text Mode of 

20 operation. Two representative algorithms for machine- 
print recognition include: WORDSCAN (Tm) sold by Calera 
Recognitions Systems of Santa Clara, California; and 
OMINPAGE (Tm) sold by Caere Corporation of Los Gatos, 
California. A separate hand-print algorithm is used by 

25 the DSP 62 to identify characters that are written on the 
display unit 14 by the pen unit 16 in a write mode of 
operation that will be discussed in greater detail below. 
One representative hand-print character recognition 
algorithm that can be employed is incorporated in the 

30 HANDWRITER (Tm) product sold by Communications 
Intelligence Corporation of Menlo Park, California. 

The DSP 62 is preferably powered down when not in 
use in order to conserve power. The DSP 62, however, 
does not contain power down data storage capability. All 

35 internal data required by the DSP 62, including OCR 



algorithms and image processing programs, must therefore 
be downloaded to the DSP 62 each time it is powered up to 
perform a processing function. The DSP 62 is configured 
such that the internal registers of the DSP 62 are 
accessed by the CPU 60 as input/ output devices over the 
bus 64. 

A real time clock 74 is also included in the 
operating system to provide a time base to support time 
of day, date, calendar and alarm functions of the 
organizer. The operation of the real time clock 74 is 
controlled by a crystal oscillator to insure accuracy and 
stability. The real time clock 74 remains operational 
when the rest of the operating system is turned off by 
the user. 

Power is supplied to the operating system via the 
power management circuit 50, which is coupled to the 
battery power unit 12 and to an emergency rechargeable 
back-up battery 72, when the main unit power ON/Off 
switch 22 is activated. The power management circuit 50 
includes a monitoring circuit that monitors the power 
level of the battery power unit 12 and switches to the 
emergency rechargeable back-up battery 72 if the 
monitored level falls below a predetermined value. A 
recharging circuit is also provided within the power 
management circuit 50 to recharge the back-up battery 72 
either from the battery power unit 12 (once a new or 
recharged battery power unit 12 is installed) or from an 
external AC or DC source that is coupled to the power 
management circuit 50 via the external power connectors 
32. The function performed by the power management 
circuit 50 is particularly important to prevent the loss 
of data stored in the RAM unit 66 which must be 
continually supplied with power. 

Data entry and retrieval is primarily accomplished 
through the high resolution touch sensitive electronic 
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display panel 14 (in conjunction with the pen input unit 
16) . A communications module 76, however, is also 
coupled to the bus 64 to permit text and image data to be 
downloaded directly to the operating system from external 
5 sources. The communication module 76, for example, 
includes a standard serial and/or parallel computer 
interface circuit (for example a standard KS232 
interface) which permits the organizer to be directly 
connected to a computer. A facsimile interface circuit 

10 and a modem are also preferably included within the 
communication module 76 to permit the organizer to 
receive and transmit data via telecommunication lines. 
To conserve space within the main unit 10, however, the 
facsimile interface and modem can be provided as separate 

15 accessory modules that are attached to the main unit 10 
when needed. 

In a preferred embodiment, an infrared 
communications link is also included in the 
communications module 76 to permit commands and data to 

20 be entered directly into and retrieved from the operating 
system of the organizer without hard-wired connections. 
The infrared communication link is particularly useful in 
providing communications between two organizers without 
having to provide a physical connection the two 

25 organizers. Thus, an individual can easily and quickly 
download schedule information or other data directly into 
a co-workers organizer. 

The touch sensitive electronic display panel 14, in 
combination with the linear electronic scanner unit 26, 

30 provides an easy to learn and use interface that permits 
the operator to enter and retrieve data from the 
organizer with a minimal amount of effort. For example. 
Fig. 6 illustrates one type of information display that 
can be presented on the display panel 14 upon power-up of 

35 the organizer in a preferred mode of operation. The 



illustrated information identifies the owner of the 
organizer by name, address, company name (with company 
logo displayed) and photograph. The advantage of the 
illustrated organizer over conventional types of 
organizers can most readily be appreciated through the 
realization that none of the information displayed in 
Fig. 6 must be manually entered into the organizer by the 
operator. 

The information displayed in Fig. 6, in contrast to 
conventional organizers, is entered through the use of 
the linear scanner unit without requiring the operator to 
key-in the text data. For example, all of the 
information is obtained simply by scanning a business 
card containing a photograph in the Photo Mode of 
operation. In such a case, the information is retained 
as a bit-map image file in the RAM unit 66 for later 
retrieval and display on the display unit 14, i.e., the 
resulting image displayed on the display unit would be an 
electronic reproduction of the original business card. 
Alternatively, the illustrated information can be 
obtained by scanning different source materials, storing 
the information in different files, and then linking the 
files together through the use of a relational database 
to retrieve and display the information on the display 
unit 14. For example, if the business card did not 
contain a photograph, the owner's name, address and 
company name could be obtained from the business card by 
scanning the business card with the scanner unit 26 in 
the Text Mode of operation. An OCR algorithm is then 
applied to the scanned image data by the DSP 62 to 
identify the text information contained therein. The 
identified text information is then stored in a primary 
database file, for example, a text based business card 
file. A photograph is then scanned in the Photo Mode of 
operation by the scanner unit 26 and the scanned 
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photographic image data is stored in a bit-map image file 
in memory. The text data is then retrieved from the 
business card file and combined with the photographic 
image data from the bit-map image file upon power-up of 
the organizer to generate the illustrated display. 

In addition to the owner information , various 
function blocks are displayed on the touch panel display 
14. The function blocks include main functions such as 
Information, Schedule and Memo, support functions such as 
Change, Find and Connect, and accessory functions such as 
Help, Tools and Options. Each of the functions are 
initiated by touching the function block with the pen 
input unit 16. For example, a daily schedule shown in 
Fig. 7 is displayed by touching the Schedule function 
block. The schedule display screen preferably takes on 
the "look and feel" of a card file. The operator can 
easily switch days by touching the "card" for the day to 
be selected. 

Information related to selected data fields 
displayed on the schedule display screen can be retrieved 
through the use of the relational database simply be 
touching the in data field with the pen input unit 16. 
For example, the illustrated schedule indicates that a 
doctor has a consult visit with a patient, Roger Brown, 
at 7:00. The doctor, however, may not remember the 
condition of the patient or may want to contact the 
patient to change the appointment time. Using 
conventional organizers, the doctor would be required to 
leave the schedule file and enter a memo file to retrieve 
the information on the patients condition and then leave 
the memo file to enter a telephone directory file to 
obtain the patient's telephone number. Switching between 
the various files is time consuming and tedious. In 
contrast, all the relevant information related to a 
selected data field is retrieved and displayed in the 



present invention as shown in Fig, 8 simply by touching 
the patient" s name with the pen input unit 16. 

The doctor may wish to obtain further information on 
the drug that has been prescribed for the patient. This 
can be accomplished by touching the Find function block 
with the pen input unit 16. A Find function menu screen 
is displayed in an overlay fashion on the display unit 14 
as shown in Fig. 9. Selecting the Look-up option causes 
a Find inquiry screen to be displayed as shown in Fig. 
10. The term to be found can be entered by simply 
touching the name of the drug on the underlying display 
screen with the pen input unit 16. The Find inquiry 
screen also asks where to look for the information. In 
the illustrated example, the "PDR Card" block is 
selected, which refers to a memory card containing 
information from the Physicians Desk Reference published 
by Medical Economic Co. Inc., of Oradell, New Jersey that 
would be located in the card interface slot 18. A search 
of the information in the PDR card is then conducted and 
the result is displayed as shown in Fig. 11. 

At this point, the doctor may wish to calculate a 
new dosage for the patient by touching the Tools function 
block to display a calculator tool screen as shown in 
Fig. 12. The "keys" of the calculator are then activated 
by touching them with a pen input unit 16. Other tool 
functions include an acoustic auto-dialer that generates 
acoustic telephone dial tones via the speaker unit 30 so 
that the doctor may use the organizer as an acoustic 
auto-dialer to call the patient. 

As was mentioned above, the ease at which related 
information can be retrieved is a direct result of the 
use of a relational database to organize all of the 
various data files to be stored in the organizer. The 
relational database is functionally illustrated in block 
diagram form in Fig. 13. All of the various files 



including a memo file, a bitmap image file, a calendar 
event file, a business card file and a telephone number 
directory file share one or more common relational 
information linking fields. The preferred relational 
information linking fields include name, address, 
telephone number , sub j ect and date • The operator is 
prompted at the initiation of a scan operation to 
identify and attach a file tag, containing one or more of 
the linking fields, to the image being scanned. The file 
tag allows each scanned image file to be easily 
identified and cross-referenced in any of the organizer's 
modes of operation. 

The entry of the file tag information, as well as 
any annotations that the operator may wish to enter on 
the scanned image, can be entered by one of two different 
methods. The first method involves the use of a virtual 
alphanumeric keyboard that is overlayed on the display 
unit 14 as shown in Fig. 14. The pen unit 16 is then 
used to select the "keys" of the virtual alphanumeric 
keyboard to enter the required information. This method, 
however, has some of the drawbacks associated with 
conventional organizers that use a stylus to activate the 
keys of a keyboard, i.e., it requires a "hunt and peck" 
type approach for those individuals that are unfamiliar 
with a standard keyboard layout. 

Fig. 15 illustrates a preferred method of entering 
data in a write mode of operation in which the operator 
can use the pen input unit 16 to print the information on 
the display unit 14. In the illustrated example, a 
diagram of a heart has been scanned from a textbook and 
displayed on the display unit 14. By activating the pen 
function block, a text input window is overlayed over the 
image of the heart. The operator then prints information 
in the blocks of the text input window using the pen 
input unit 16. The DSP 62 applies a hand-print text OCR 
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algorithm to identify the text characters that were 
printed in the blocks. 

Text information can also be transferred from 
scanned images directly into a text data file without 
5 requiring the operator to key-in the text data. Fig. 16, 
for example, illustrates a text information file that is 
displayed on the display unit 14 which " contains a 
patient's personal information as well as information on 
the patient's medical insurance. The medical insurance 
10 information for the text information file is obtained 
directly from the patient's scanned medical card, the 
image of which is illustrated in Fig. 17. The CPU 60 
performs a text identification routine to a bit map of 
the scanned medical card to identify areas of the bit map 
15 that contain text information. A box is drawn around 
each of the areas that are determined to contain text 
information. The DSP 62 then performs an OCR text 
recognition algorithm to the data contained within the 
areas specified by the boxes to identify the text data 
50 contained within the boxes. The operator can then 
transfer the identified text data within selected boxes 
into the text information file by touching a selected box 
to fill in a template field that is overlayed on the 
display. The template field continues to prompt the user 
5 to select a box for each of the fields in the text 
information file. 

The above-described scanning operations are further 
illustrated in flow diagram form in Figs. 18 and 19. 
Fig. 18 illustrates the operation of the scanner unit 26 
D and DSP 62 to scan an image. At step SI, the operator 
selects which type of scanning mode (either the Text Mode 
or the Photo mode) is to be employed. After selection of 
the scanning mode, instructions are displayed on the 
display unit 14 at step S2 to tell the operator how to 
> perform the scanning operation. The CPU 60 then enters 
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a wait state to wait form the activation of the start 
scan button by the operation at step S3- Once the start 
scan button is activated, the CPU 60 turns the display 
unit 14 off to conserve power at step S4 and then applies 
power to the linear scanning unit at step S5. The CPU 60 
then activates the DSP 62 at step S6 and loads the DSP 62 
with the appropriate OCR program based on the type of 
scanning mode selected by the user. Digitized image data 
is downloaded from the image sensor to the DSP 62 at step 
S7 and the DSP 62 processes the image data and stores the 
result in the RAM unit 66 at step SB. The CPU 60 turns 
off the DSP 62 and the linear scanning unit once all the 
image data is processed at step S9 scan lines have been 
entered. 

Fig. 19 illustrates the entry of data in the write 
mode of operation in the text input window using the pen 
input unit 16. At step SI, the DSP 62 is powered up and 
then loaded with the hand-print OCR software from the ROM 
unit. At step S2-S3, the DSP 62 is placed in a standby 
mode and the text entry window is displayed on the 
display unit 14. At step S4, the operator uses the pen 
input unit 16 to write a character in a block of the text 
entry window. At step S5 f the DSP 62 is removed from the 
standby mode and a bit-map representation of the hand- 
print character is passed to the DSP 62 for processing. 
The DSP 62 applies the free-hand OCR algorithm to the 
bit-map representation at to determine the text character 
represented by the bit-map representation. The 
identified text character is displayed above the block of 
the text entry window at step S6 for verification by the 
operator at step S7. The character is stored in memory 
if it has been correctly identified at step S8. If the 
character has not been correctly identified an error 
message is displayed at step S9 and the operator is 
required to re-enter the character. 
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A more complete understanding of the operation of 
the organizer can be gained through detailed study of the 
attached Appendix which forms a part of this 
specification. The attached Appendix contains a detailed 
5 program listing of a program which demonstrates the 
interaction of the various organizer functions described 
above. The demonstration program is written in the "C ,f 
programming language and is intended to be executed on a 
computer loaded with the BORLAND (Tm) C compiler. 
10 Section A of the appendix contains the main operating 
program "grScreen". The remaining sections of the 
appendix contain various program modules, target tables, 
parameters and definitions that are required to run 
grScreen. 

15 It should be noted that the Appendix contains 

material that is subject to copyright protection. The 
copyright owner has no objection to the facsimile 
reproduction of this specification as it appears in the 
files of the United States Patent and Trademark Office 
20 for the purposes of studying the invention, but otherwise 
reserves all copyrights whatsoever. 

The above described image based electronic organizer 
provides an easy to learn and use interface, through the 
combination of the touch sensitive display unit 14, pen 
25 input unit 16 and linear scanning unit, which reduces the 
amount of effort required to enter and retrieve data into 
the electronic organizer. It further provides the 
ability to correlate information from several different 
sources to permit related information to be quickly and 
30 easily reviewed with a minimal amount of effort through 
the use of a relational database. The organizer can also 
be customized to fit the needs of a variety of business 
professions. 

The utility of the organizer can further be extended 
35 through the use of various accessory devices. Fig- 20, 
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for example, illustrates a docking station 70 that is 
designed to hold the organizer when the operator is at a 
desk or workstation. In the illustrated example, the 
main unit 10 of the organizer is provided with a docking 
5 connector 72 that mates with a corresponding docking 
connector (not shown) located on the docking station 70. 
The docking station 70 includes additional " memory card 
expansion slots 74 that are accessible by the main unit 
10 when the organizer is located in the docking station 

10 70. Alternatively, memory cards located in the expansion 
slots 74 can be used to store data received from a 
facsimile conversion module 76, modem module 78 or 
personal computer 80 (PC) coupled to the docking station 
70 when the main unit 10 is not located in the docking 

15 station 70. The memory cards are then transferred from 
the docking station 70 to the main unit 10 to be 
accessed. 

The docking station 70 further includes a recharge 
station 74 which is used to charge an additional battery 

20 unit 12' that replaces the battery unit 12 when it is 
discharged. The main unit 10, however, does not use 
battery power when located in the docking station, but 
instead draws power from a power supply located within 
the docking station 70. Thus, the battery units 12, 12 1 

25 can be recharged while the organizer is located in the 
docking station 70. Additional accessories , such as a 
keyboard 82 can also be plugged into the docking station 
70 for interaction with the main unit 10 of the 
organizer. 

30 The invention has been described with reference to 

certain preferred embodiments thereof. It will be 
understood, however, that modifications and variations 
are possible. Fig. 21, for example, illustrates an 
organizer in accordance with the invention that includes 

35 an electronic camera unit having a lens 90 that focuses 
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the image of a subject onto an electronic imaging device 
(not shown) provided within the body of the main unit 10. 
A shutter release button 92 is provided to activate 
camera control circuitry (not shown) coupled to the 
5 electronic imaging device and the bus 64. 

In a preferred mode of operation, partial activation 
of the shutter release button 92 causes the* output from 
the electronic imaging device to be supplied to the 
display unit 14 so that the display unit 14 can be used 
10 as an active "viewf inder". Once the image of the subject 
has been properly framed on the display unit 14, the 
shutter release button 92 is fully depressed in order to 
capture the image of the subject. Image data from the 
electronic camera unit is stored as a bit map image in 
15 the same manner as image data received from the linear 
electronic scanning unit 26. 

The electronic camera unit can be used in 
combination with, or in place of, the electronic scanning 
unit 26 to enter data into the organizer. When used in 
20 combination, the electronic camera unit is preferably 
provided as an accessory module that couples to the 
docking connector 72 provided on the main unit 10 (See 
Fig. 20). The provision of a separate accessory module 
for the electronic camera unit enables the overall size 
25 of the main unit 10 to be kept to a minimum. 

Other structural variations are also possible other 
than those specifically set forth above. For example, 
the small document transport mechanism can be configured 
such that it does not extend from the "main unit 10, but 
30 instead, includes a document entry slot on the side of 
the main unit 10, a document path within the main unit 10 
that passes the scanning unit, a document pinch roller 
driven by the transport wheels in the document path, and 
a document entry slot past the point of the scanning 
35 unit. 
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Industrial Utility 
The invention provides an electronic pocket 
organizer that is useful in storing information that must 
be accessed by individual's on a routine basis. The 
5 advantages of the organizer are that is can be adapted 
for a variety of specific uses by modifying the 
organizer 1 s system software; the user interface employed 
in the organizer simplifies the entry of both text and 
image data into the electronic organizer; data is stored 

10 in the organizer in a relational database format that 
further enhances organizer operation by permitting- 
related information to be quickly and easily entered and 
retrieved from a plurality of databases; and a small 
document transport mechanism incorporated in the 

15 organizer permits small documents to be transported past 
the linear scanning unit of the organizer without using 
energy from the organizer's battery unit. The small 
document transport mechanism, while particularly well 
suited for use in the organizer, can also be incorporated 

20 in any type of hand-held scanner. 
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WHAT IS CLAIMED IS: 

1. An electronic organizer characterized by: 
text data entry means for entering text data into a 
memory unit; image data entry means for entering image 
data in the memory unit; processing means for 
retrieving the text and image dat& entered. in the 
memory unit; and display means for displaying the 
retrieved text and image data. 

2. An electronic organizer as claimed in 
claim 1, wherein the image data entry means includes a 
linear electronic scanning unit and the display means 
includes a touch sensitive display unit. 

3. An electronic organizer as claimed in 
claim 1, wherein the text and image data are stored in 
the memory unit in a relational database format. 

4. An electronic organizer as claimed in 
claim 2, further comprising a small document scanner 
mechanism for transporting small documents past the 
electronic scanning unit. 

5. An electronic organizer characterized by: 
a main unit including an integral scanning unit, a 
touch sensitive display unit, a memory unit, and 
control means for controlling the processing and 
storage of text and image data entered into the main 
unit through the integral scanning unit and the touch 
sensitive display unit; a battery unit releaseably 
coupled to the main unit; and an operator control for 
activating the main unit. 
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6. An electronic organizer as claimed in 
claim 5, wherein the main unit further comprising a 

. small document transport mechanism. 

7. An electronic organizer as claimed in 
claim 6, wherein the small document transport mechanism 
extends from the main unit during operation. 

8. An electronic organizer as claimed in 
claim 6, wherein the small document transport mechanism 
characterizes transport wheels coupled to a drive axle 
and a document pinch roller coupled to the drive axle 
by pinch roller drive gears . 

9 . An electronic organizer as claimed in 
claim 8, further includes an encoder unit coupled to 
the drive axle by an encoder gear assembly, wherein the 
encoder unit generates signals, indicative of the 
movement of the organizer during a scanning operation, 
that are supplied to the control means. 

10. An electronic organizer as claimed in 
claim 5, wherein machine generated text data is scanned 
by the scanning unit to produce scanned text data that 
is supplied to the control means, and wherein the 
control means applies an optical character recognition 
routine to the scanned text data produced by the 
scanning unit to convert the scanned text data to 
computer coded text data. 

11. An electronic organizer as claimed in 
claim 10, wherein hand-printed text data is produced by 
the touch sensitive display unit in response to an 
operator input and supplied to the control means, and 
wherein the control means performs and optical 
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character recognition routine on the hand-printed text 
data to generate computer coded text data, 

12. An electronic organizer as claimed in 
claim 5, wherein the image and text data is stored in 
the memory unit in a relational database format in 
which a file tag, containing at least one linking 
field, is used to identify related image and text data 
in a plurality of databases. 

13. An electronic organizer as claimed in 
claim 5, wherein the control means includes a central 
processing unit and a digital signal processing unit 
coupled to a system bus, the display unit and the 
memory unit are coupled to the system bus, and the 
scanning unit is coupled to the digital signal 
processing unit. • 

14. An electronic organizer as claimed in 
claim 5, wherein the main unit further includes a 
speaker unit coupled to the control means. 

15. An electronic organizer as claimed in 
claim 5, wherein the main unit further includes a 
communications module coupled to the control means. 

16. An electronic organizer as claimed in 
claim 5, further comprising a pen unit that is used to 
activate the touch sensitive display unit. 

17. An electronic organizer as claimed in 
claim 7 , wherein the main unit further includes memory 
expansion slots coupled to the system bus. 
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18. An electronic organizer as claimed in 
claim 5, further includes an electronic camera unit 
coupled to the control means of the main unit. 

19. An electronic organizer characterized 
by: a central processing unit, a digital signal 
processing unit, a random access memory. unit, a read 
only memory unit, a real time clock and memory 
expansion slots coupled to the system bus; a linear 
scanning unit coupled to the digital signal processing 
unit; a touch sensitive display unit coupled to the 
system bus; and a power management unit coupled to the 
system bus, a rechargeable primary battery unit and a 
rechargeable back-up battery unit. 
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